System for monitoring back seat in transporter and method using the same

ABSTRACT

Disclosed are systems and a methods for monitoring a back seat in a transporter based on a head up display (HUD), the system a back seat display mounted on the back seat, and the HUD to display monitoring information selected by a user, image information filmed by a camera, and control information for an image displayed on the back seat display, and a processor configured to transmit and receive a control signal between the camera, the back seat display, and the HUD, and control zoom-in or zoom-out of a first area of the HUD where the monitoring information is displayed to not overlap a second area where driving control information is displayed, wherein the image information comprises an image content displayed on the back seat display, a reproduction time of the image content, and a number or a reproduction time of image contents after a point in time.

CROSS REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit under 35 USC § 119(a) of KoreanPatent Application No. 10-2021-0175580, filed on Dec. 9, 2021, theentire disclosure of which is incorporated herein by reference for allpurposes.

BACKGROUND Field

The present disclosure relates to a system for monitoring a back seat ina transporter and a method using the same, and specifically, to a systemdesigned to efficiently monitor a situation of the back seat in whichinfants sits in the transporter based on a head up display (HUD) and tointeract with the monitored situation, and a method using the same.

Discussion of Related Art

When driving a vehicle with a child, parents are usually located in adriver's seat and a passenger seat, and the child is generally locatedin a back seat. When only one of the parents is in the driver's seat,there is a great demand for a function of identifying a behavior of thechild in the back seat in real time while driving and controlling acontent reproduced by the child.

Previous studies on the same have provided a function of displaying anoccupant's state monitoring image on a display at a position of a centerfascia of the vehicle. However, in such previous studies, there is aproblem in that a driver is distracted as the image of the back seatoccupant is displayed only on the display at the center fascia position,and it is necessary to improve a method for efficiently controlling theback seat occupant's image such that there is no problem in the driving.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

In one general aspect, there is provided an apparatus for monitoring aback seat in a transporter, the apparatus including a camera and a backseat display mounted on the back seat of the transporter, a head updisplay (HUD) configured to display any one or any combination ofmonitoring information selected by a user from among image informationdisplayed on the back seat display, image information filmed by thecamera, and control information for an image displayed on the back seatdisplay, and a processor configured to transmit and receive a controlsignal between the camera, the back seat display, and the HUD, andcontrol zoom-in or zoom-out of a first area of the HUD where themonitoring information is displayed to not overlap a second area wheredriving control information is displayed by the HUD, wherein the imageinformation displayed on the back seat display comprises any one or anycombination of an image content displayed on the back seat display, areproduction time of the image content, and a number or a reproductiontime of image contents reproduced on the back seat display after a pointin time.

The HUD may be an augmented reality HUD (AR-HUD), wherein a brightnessand a color of the any one or any combination of monitoring informationmay be controlled to minimize an effect on monitoring of a frontsituation.

The processor may be configured to control the control information forthe image displayed on the back seat display to be displayed on the HUDbased on a preset criterion of the user, in response to the imagedisplayed on the back seat display corresponding to a category, whereinthe control information for the image displayed on the back seat displaymay include control information for identifying any one or anycombination of reproduction, stop, and next content reproduction of animage content waiting to be displayed on the back seat display.

The processor may be configured to control a notification to be providedas the control information for the image displayed on the back seatdisplay, in response to the number or the total reproduction time ofimage contents reproduced on the back seat display exceeds a threshold.

The image information filmed by the camera may include any one or anycombination of determination result information based on a heart rate, abody temperature, and a posture of a back seat occupant along with animage of the back seat occupant.

The image information filmed by the camera may additionally containsimage information of at least one of an object and a pet located in theback seat, wherein the processor may be configured to highlight an imagearea of the object or the pet recognized by the camera based on an inputof the user.

The apparatus may include a voice sensor mounted on the back seat of thetransporter, and a front camera mounted at a position to film an imageof a driver of the transporter, wherein the processor may be configuredto control the image of the driver to be displayed on the back seatdisplay in response to a specific voice recognized by the voice sensor.

In another general aspect, there is provided a processor-implementedmethod for monitoring a back seat in a transporter, the method includingacquiring, via a camera mounted on the back seat of the transporter, aback seat image containing a back seat occupant, displaying, on a headup display (HUD), any one or any combination of monitoring informationselected by a user from among image information displayed on a back seatdisplay mounted on the back seat of the transporter, the back seatimage, and control information for an image displayed on the back seatdisplay, and performing zoom-in or zoom-out of a first area where themonitoring information is displayed on the HUD to not overlap a secondarea where driving control information is displayed on the HUD, whereinthe image information displayed on the back seat display comprises anyone or any combination of an image content being displayed on the backseat display, a reproduction time of the image content, and a number ora reproduction time of image contents reproduced on the back seatdisplay after a point in time.

The HUD may be an augmented reality HUD (AR-HUD), wherein a brightnessand a color of the any one or any combination of monitoring informationmay be controlled to minimize an effect on monitoring of a frontsituation.

The method may include controlling the control information for the imagedisplayed on the back seat display to be displayed on the HUD, inresponse to the image displayed on the back seat display correspondingto a category specified by the user, wherein the control information forthe image displayed on the back seat display may include controlinformation for identifying any one or any combination of reproduction,stop, and next content reproduction of an image content waiting to bedisplayed on the back seat display.

The method may include providing a notification as the controlinformation for the image displayed on the back seat display, inresponse to the number or the total reproduction time of image contentsreproduced on the back seat display exceeding a threshold specified bythe user.

The image information filmed by the camera may include any one or anycombination of determination result information based on a heart rate, abody temperature, and a posture of the back seat occupant along with animage of the back seat occupant.

The image information filmed by the camera may additionally containimage information of at least one of an object and a pet located in theback seat, the method may include highlighting an image area of theobject or the pet recognized by the camera based on an input of theuser.

The method may include displaying an image of a driver on the back seatdisplay in response to a voice being recognized by a voice sensormounted on the back seat of the transporter.

Other features and aspects will be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1 and 2 are diagrams for illustrating a concept of a HUD-basedback seat monitoring system according to an embodiment of the presentdisclosure.

FIG. 3 is a diagram for illustrating a configuration of a HUD-based backseat monitoring system according to an embodiment of the presentdisclosure.

FIG. 4 is a diagram for illustrating a scheme for a processor to controlHUD display information according to an embodiment of the presentdisclosure.

FIG. 5 is a diagram for illustrating a method for controlling an imagecontent reproduced by a back seat display according to an embodiment ofthe present disclosure.

FIG. 6 is a diagram for illustrating a method for efficientlycommunicating with a driver based on voice recognition of a back seatoccupant according to an embodiment of the present disclosure.

FIG. 7 is a diagram for illustrating a method for monitoring an objectand a pet by utilizing a back seat camera according to an embodiment ofthe present disclosure.

FIG. 8 is a diagram for illustrating a HUD-based in-transporter backseat monitoring method according to an embodiment of the presentdisclosure.

Throughout the drawings and the detailed description, unless otherwisedescribed or provided, the same drawing reference numerals will beunderstood to refer to the same elements, features, and structures. Thedrawings may not be to scale, and the relative size, proportions, anddepiction of elements in the drawings may be exaggerated for clarity,illustration, and convenience.

DETAILED DESCRIPTION

The following detailed description is provided to assist the reader ingaining a comprehensive understanding of the methods, apparatuses,and/or systems described herein. However, various changes,modifications, and equivalents of the methods, apparatuses, and/orsystems described herein will be apparent after an understanding of thedisclosure of this application. For example, the sequences of operationsdescribed herein are merely examples, and are not limited to those setforth herein, but may be changed as will be apparent after anunderstanding of the disclosure of this application, with the exceptionof operations necessarily occurring in a certain order.

The features described herein may be embodied in different forms and arenot to be construed as being limited to the examples described herein.Rather, the examples described herein have been provided merely toillustrate some of the many possible ways of implementing the methods,apparatuses, and/or systems described herein that will be apparent afteran understanding of the disclosure of this application.

Throughout the specification, when a component “includes” a certaincomponent, it means that other elements may be further included, ratherthan excluding other components, unless otherwise stated.

In addition, a ‘transporter’ used in the following description assumes avehicle used for transport of an occupant, but is not limited thereto.The ‘transporter’ may include all of an urban air mobility (UAM), a bus,and the like.

FIGS. 1 and 2 are diagrams for illustrating a concept of a HUD-basedback seat monitoring system according to an embodiment of the presentdisclosure.

First, an in-transporter back seat monitoring system according to anembodiment shown in FIG. 1 includes a camera and a back seat display 120mounted on a back seat of a transporter. As shown in FIG. 2 , a camera130 may be integrally mounted on the back seat display 120, but thepresent disclosure may not be necessarily limited thereto. The camera130 may be installed at various positions in the back seat where a stateof a monitoring target may be filmed.

In addition, the back seat monitoring system according to the presentembodiment includes a HUD that displays at least one monitoringinformation selected by a user among image information 1108 displayed onthe back seat display 120 described above, image information 110C (e.g.,an image showing a behavior of an infant in a back seat) filmed by thecamera described above, and control information on the image displayedon the back seat display (not shown; e.g., information for realizing acontrol function for reproduction and stop of the corresponding imageand next content reproduction). It is preferable that the HUD has a formof an AR-HUD in which control information 110A related to travel isdisplayed with minimum obstruction of a view in an actual frontsituation, but the present disclosure is not limited thereto.

As shown in FIG. 1 , as the HUD, preferably the AR-HUD, rather than adisplay located at a center fascia is utilized for the back seatmonitoring, obstruction of a view of a parent who is a driver during thetravel may be minimized and the control may be performed in an efficientmanner.

FIG. 3 is a diagram for illustrating a configuration of a HUD-based backseat monitoring system according to an embodiment of the presentdisclosure.

As described above in relation to FIGS. 1 and 2 , the in-transporterback seat monitoring system according to the present embodiment includesthe back seat display 120 and the back seat camera 130 on the back seatto allow a state of the occupant such as the infants in the back seatand a content being reproduced to be identified. In addition, in onepreferred embodiment, in addition to the back seat display 120 and theback seat camera 130 described above, a (back seat) voice sensor 140 isadditionally included for communication with the driver via voicerecognition of the occupant such as the infants located in the backseat.

In one example, a HUD 160 is disposed on a front face of the transporteras described above to allow the at least one monitoring informationselected by the user among the image information displayed on the backseat display 120, the image information filmed by the back seat camera130, and the control information on the image displayed on the back seatdisplay 120 to be displayed. In addition, in a preferred embodiment ofthe present disclosure, a front camera 170 is additionally disposed toprovide means to communicate with the driver based on the voicerecognition of the occupant such as the infants located in the backseat. This will be described later in detail with reference to FIG. 6 .

In one example, the back seat camera 130, the back seat display 120, thevoice sensor 140, the HUD 160, and the front camera 170 described abovemay be connected to a processor 150 and operate in response to a controlsignal of the processor 150.

FIG. 4 is a diagram for illustrating a scheme for a processor to controlHUD display information according to an embodiment of the presentdisclosure.

As shown in FIG. 4 , the processor may display, as information displayedby the HUD, a combination of the image content 1108 displayed by theback seat display, the state image 110C of the back seat occupantsecured by the back seat camera, information 330 on the image contentdisplayed by the display, and the like based on the selection of theuser. It is desirable to control a first area 310 in which suchmonitoring information is displayed not to overlap with a second area320 in which driving control information displayed by the HUD isdisplayed.

In the first area 310 in which the monitoring information is displayed,the processor may freely zoom in and out based on the user's selection,but such zoom in/zoom out control is performed within a range that doesnot overlap with the second area 320. Preferably, the first area 310 maybe located at a lower left end of the driver's seat as in an exampleshown in FIG. 4 , but the present disclosure may not be necessarilylimited thereto.

In addition, the image content 1108 displayed by the back seat display,the state image 110C of the back seat occupant secured by the back seatcamera, and the like displayed by the AR-HUD are not for viewing thebehavior of the occupant in high quality, but for the monitoring, sothat it is preferable that the images are displayed with a level ofbrightness/color similar to that of another driving control information320 displayed via the AR-HUD while minimizing the obstruction of theview while driving. A brightness/color/size/position of the imagecontent 1108 displayed by the back seat display, the state image 110C ofthe back seat occupant secured by the back seat camera, and the likedisplayed by the AR-HUD may be adjusted in advance by the user'sselection.

In addition, the information 330 on the image content displayed by thedisplay may contain at least one of a reproduction time of the imagecontent and the number or a total reproduction time of image contentsreproduced on the back seat display after a specific time point (e.g.,after start traveling), as illustrated in FIG. 4 .

In addition, the image information 110C filmed by the back seat cameramay contain at least one of determination result information based on aheart rate, a body temperature, and a posture of the back seat occupantalong with the image of the back seat occupant.

FIG. 5 is a diagram for illustrating a method for controlling an imagecontent reproduced by a back seat display according to an embodiment ofthe present disclosure.

In the present embodiment, it is suggested to automatically warn thedriver and activate a control function based on presetting of the userwhen the image displayed on the back seat display corresponds to aspecific category. For example, as shown in FIG. 5 , when it isrecognized that a content 1108 inappropriate for the infants isreproduced on the back seat display while the infant is in the back seator when a total image watching time or the number of image watching 330exceeds a preset criterion, the system according to the presentembodiment may be set to automatically provide a notification (e.g.,screen display or voice notification) to the driver and display controlinformation 410 for the image displayed on the back seat display.

The control information 410 at this time may be control information foridentifying at least one of reproduction, stop, and next contentreproduction of an image content waiting to be displayed on the backseat display. Although the control information 410 displayed on the HUDis assumed in FIG. 5 , but the present disclosure is not necessarilylimited thereto. The control information 410 may be implemented via aninterface of the voice notification/voice command.

FIG. 6 is a diagram for illustrating a method for efficientlycommunicating with a driver based on voice recognition of a back seatoccupant according to an embodiment of the present disclosure.

As described above in relation to FIG. 3 , the back seat monitoringsystem according to the present embodiment may additionally include thevoice sensor 140 mounted on the back seat of the transporter, and thefront camera 170 that is mounted at a position where the image of thedriver of the transporter may be filmed. Therefore, in response to aspecific voice (e.g., Thom′ and ‘dad’) of the back seat occupantrecognized by the voice sensor 140, as shown in (b) in FIG. 6 , it maybe controlled that the driver's image is displayed on the back seatdisplay 120.

Because an image of the infant in the back seat is displayed via the HUD((A) in FIG. 6 ), the image of the driver may be displayed on the backseat display 120 for the back seat occupant, so that the driver and theinfant may communicate with each other while identifying a behavior ofeach other during the driving.

FIG. 7 is a diagram for illustrating a method for monitoring an objectand a pet by utilizing a back seat camera according to an embodiment ofthe present disclosure.

In the above description, it has been assumed that the image filmed bythe back seat camera is the behavior of the back seat occupant such asthe infants, but the present disclosure is not necessarily limitedthereto. A function of additionally monitoring an object, a pet, and thelike in the back seat via the back seat camera may be provided.

For example, as shown in FIG. 7 , an area of an infant 710 who is theback seat occupant and an area of an object 720 located in the back seatin the image filmed by the back seat camera may be recognized.Therefore, when the driver wants to identify whether a specific object(e.g., a wallet) is located in the back seat while driving, the objectlocated in the back seat may be identified by highlighting therecognized object as shown in FIG. 7 . In addition, in order to minimizethe distraction during the driving, the identification of the object maybe guided to the driver via voice.

In addition, when the pet is located in the back seat, a behavior of thepet may be monitored by applying the above-described embodiments as itis, and postures of the back seat occupant and the pet may be monitoredand notification may be provided via the AR-HUD.

FIG. 8 is a diagram for illustrating a HUD-based in-transporter backseat monitoring method according to an embodiment of the presentdisclosure.

As described above, the back seat monitoring method according to thepresent embodiment first acquires the back seat image containing theback seat occupant via the camera mounted on the back seat of thetransporter (S810). It is assumed that the back seat image is the imageof the back seat occupant such as the infant, but the present disclosureis not necessarily limited thereto. As described above with reference toFIG. 7 , the images of the pet and the object in the back seat may beincluded.

Therefore, the at least one monitoring information selected by the useramong the image information displayed on the back seat display mountedon the back seat of the transporter, the above-described back seatimage, and the control information for the image displayed on the backseat display may be displayed on the HUD (S820). It is preferable thatthe HUD is the AR-HUD and it is preferable that such display informationis displayed as AR information that minimizes the obstruction of thedriving, unlike image display for image watching.

In addition, the zoom-in or the zoom-out of the first area where themonitoring information displayed by the HUD is displayed is performed,and it is preferable that the first area is controlled so as not tooverlap the second area where the driving control information displayedby the HUD is displayed (S830).

The control of the image displayed on the back seat display may beefficiently performed while driving via the information displayed assuch as shown in FIG. 8 (S831), and the driver and the infant maycommunicate with each other while identifying the behavior of each otherin response to the specific voice of the back seat occupant as describedabove in relation to FIG. 6 (S832). In addition, not only the monitoringof the back seat occupant, but also the monitoring of the object and thepet may be performed (S833), and for example, whether there is aspecific object in the back seat may be efficiently identified duringthe travel.

The system and the method for monitoring the back seat in thetransporter according to the embodiments of the present disclosure asdescribed above may be utilized for the driver who is with the infant toprovide convenience, and may also be utilized for the pet, objectmanagement, and the like according to the embodiment.

Described above is a system that efficiently displays informationrelated to an occupant such as infants via an augmented reality head updisplay (AR-HUD) without distracting a driver for travel safety, and amethod using the same.

In addition, the system described above may efficiently control acontent of a back seat display while driving and efficiently performcommunication with an occupant located in the back seat, recognition ofan object and a pet, and the like, and a method using the same.

In addition, the system described above may monitor the occupant such asthe infants may be efficiently performed while ensuring the travelsafety without distracting the driver.

In addition, the content of the back seat display may be efficientlycontrolled while driving, and the communication with the occupantlocated in the back seat, the recognition of the object and the pet, andthe like may be efficiently performed.

While this disclosure includes specific examples, it will be apparentafter an understanding of the disclosure of this application thatvarious changes in form and details may be made in these exampleswithout departing from the spirit and scope of the claims and theirequivalents. The examples described herein are to be considered in adescriptive sense only, and not for purposes of limitation. Descriptionsof features or aspects in each example are to be considered as beingapplicable to similar features or aspects in other examples. Suitableresults may be achieved if the described techniques are performed in adifferent order, and/or if components in a described system,architecture, device, or circuit are combined in a different manner,and/or replaced or supplemented by other components or theirequivalents.

Therefore, the scope of the disclosure is defined not by the detaileddescription, but by the claims and their equivalents, and all variationswithin the scope of the claims and their equivalents are to be construedas being included in the disclosure.

What is claimed is:
 1. An apparatus for monitoring a back seat in atransporter, the apparatus comprising: a camera and a back seat displaymounted on the back seat of the transporter; a head up display (HUD)configured to display any one or any combination of monitoringinformation selected by a user from among image information displayed onthe back seat display, image information filmed by the camera, andcontrol information for an image displayed on the back seat display; anda processor configured to transmit and receive a control signal betweenthe camera, the back seat display, and the HUD, and control zoom-in orzoom-out of a first area of the HUD where the monitoring information isdisplayed to not overlap a second area where driving control informationis displayed by the HUD, wherein the image information displayed on theback seat display comprises any one or any combination of an imagecontent displayed on the back seat display, a reproduction time of theimage content, and a number or a reproduction time of image contentsreproduced on the back seat display after a point in time.
 2. Theapparatus of claim 1, wherein the HUD is an augmented reality HUD(AR-HUD), wherein a brightness and a color of the any one or anycombination of monitoring information is controlled to minimize aneffect on monitoring of a front situation.
 3. The apparatus of claim 1,wherein the processor is further configured to control the controlinformation for the image displayed on the back seat display to bedisplayed on the HUD based on a preset criterion of the user, inresponse to the image displayed on the back seat display correspondingto a category, wherein the control information for the image displayedon the back seat display comprises control information for identifyingany one or any combination of reproduction, stop, and next contentreproduction of an image content waiting to be displayed on the backseat display.
 4. The apparatus of claim 1, wherein the processor isfurther configured to control a notification to be provided as thecontrol information for the image displayed on the back seat display, inresponse to the number or the total reproduction time of image contentsreproduced on the back seat display exceeds a threshold.
 5. Theapparatus of claim 1, wherein the image information filmed by the cameracomprises any one or any combination of determination result informationbased on a heart rate, a body temperature, and a posture of a back seatoccupant along with an image of the back seat occupant.
 6. The apparatusof claim 5, wherein the image information filmed by the cameraadditionally contains image information of at least one of an object anda pet located in the back seat, wherein the processor is furtherconfigured to highlight an image area of the object or the petrecognized by the camera based on an input of the user.
 7. The apparatusof claim 1, further comprising: a voice sensor mounted on the back seatof the transporter; and a front camera mounted at a position to film animage of a driver of the transporter, wherein the processor is furtherconfigured to control the image of the driver to be displayed on theback seat display in response to a specific voice recognized by thevoice sensor.
 8. A processor-implemented method for monitoring a backseat in a transporter, the method comprising: acquiring, via a cameramounted on the back seat of the transporter, a back seat imagecontaining a back seat occupant; displaying, on a head up display (HUD),any one or any combination of monitoring information selected by a userfrom among image information displayed on a back seat display mounted onthe back seat of the transporter, the back seat image, and controlinformation for an image displayed on the back seat display; andperforming zoom-in or zoom-out of a first area where the monitoringinformation is displayed on the HUD to not overlap a second area wheredriving control information is displayed on the HUD, wherein the imageinformation displayed on the back seat display comprises any one or anycombination of an image content being displayed on the back seatdisplay, a reproduction time of the image content, and a number or areproduction time of image contents reproduced on the back seat displayafter a point in time.
 9. The method of claim 8, wherein the HUD is anaugmented reality HUD (AR-HUD), wherein a brightness and a color of theany one or any combination of monitoring information is controlled tominimize an effect on monitoring of a front situation.
 10. The method ofclaim 8, further comprising: controlling the control information for theimage displayed on the back seat display to be displayed on the HUD, inresponse to the image displayed on the back seat display correspondingto a category specified by the user, wherein the control information forthe image displayed on the back seat display comprises controlinformation for identifying any one or any combination of reproduction,stop, and next content reproduction of an image content waiting to bedisplayed on the back seat display.
 11. The method of claim 10, furthercomprising: providing a notification as the control information for theimage displayed on the back seat display, in response to the number orthe total reproduction time of image contents reproduced on the backseat display exceeding a threshold specified by the user.
 12. The methodof claim 8, wherein the image information filmed by the camera comprisesany one or any combination of determination result information based ona heart rate, a body temperature, and a posture of the back seatoccupant along with an image of the back seat occupant.
 13. The methodof claim 12, wherein the image information filmed by the cameraadditionally contains image information of at least one of an object anda pet located in the back seat, wherein the method further comprises:highlighting an image area of the object or the pet recognized by thecamera based on an input of the user.
 14. The method of claim 8, furthercomprising displaying an image of a driver on the back seat display inresponse to a voice being recognized by a voice sensor mounted on theback seat of the transporter.